Image restoration algorithms often require previous knowledge about the point spread function (PSF) of the disturbance.
Some good examples of the term PSF follow in the illustrations of the present patent application and their accompanying explanations.
Deriving the PSF manually from a degraded ideal step-edge in the image is a well known procedure intended mainly for isotropic degradations. A common image degradation that can be approximated as isotropic is the atmospheric blurring in long distance imaging.
U.S. Pat. No. 7,162,100, “Methods for image enhancement”, of Gregory, discloses an improvement on a known method, Richardson Lucy (RL) deconvolution for restoration of blurred images, by means of an algorithm including several iterations. The patent teaches a method of restoring the image on the assumption of a known PSF; however has a major disadvantage in not teaching how to estimate the PSF solely (and automatically) from the data available in the image.
U.S. Pat. No. 5,841,911, “Method for the restoration of images disturbed by the atmosphere”, of Kopeika et al., discloses a method including direct blind deconvolution, however one of the disadvantages of this method is that it is not general, and finding the atmospheric modulation transfer function (MTF) is done according to weather data, requiring a meteorological station alongside the imaging system.
U.S. Pat. No. 6,859,564 “Signal processing using the self de-convolving data reconstruction algorithm”, of Caron discloses the description of an automatic method for the improvement of images subject to blur, however one of its major disadvantages is assumption that PSF depends on the smoothed magnitude of the true image, while in fact, the PSF characterizes degradation causes which are unrelated to the true image, as the same true image can be blurred by many different PSF types.
The present invention may be better understood with reference to the following scientific papers:
1. Kopeika, N. S., A System Engineering Approach to Imaging, second ed. SPIE Optical Engineering Press, Bellingham, Wash., (1998).
2. Jalobeanu, A., Zerubia, J., Blanc-Feraud, L., Bayesian estimation of blur and noise in remote sensing imaging, Campisi, P., Egiazarian, K. (Eds.), Blind image deconvolution: theory and applications. Taylor & Francis/CRC press, May, (2007).
3. Kundur, D., Hatzinakos, D., Blind image deconvolution, IEEE Signal Processing Mag. 13, 43-64, (1996).
4. Carasso, A., Direct blind deconvolution, SIAM Journal of Applied Math 61(6), 1980-2007, (2001).
5. Chalmond, B., PSF estimation for image deblurring, Graphical models and Image Processing 53(4), 364-372, (1991).
6. Yitzhaky, Y., Milberg, R., Yohayev, S., Kopeika, N. S., Comparison of direct blind deconvolution methods for motion-blurred images, Appl. Opt. 38(20), 4325-4332, (1999).
7. Nishi, K., Ando, S., Blind superresolving image recovery from blur invariant edges, Proc. of Int. Conf. Acoustics, Speech, Signal Processing. Vol. 5. Adelaide, Australia, pp. 85-88, (1994).
8. Pavlovic, G., Tekalp, A. M., Maximum Likelihood Parametric Blur Identification Based on a Continuous Spatial Domain Model, IEEE Trans. Image Process. 1(4), 496-504, (1992).
9. Savakis, A. E., Trussell, H. J., Blur identification by residual spectral matching. IEEE Trans, Image Process. 2(2), 141-151, (1993).
10. Sheppard, D. G., Bobby, H., Michael, M., Iterative multi-frame super-resolution algorithms for atmospheric-turbulence-degraded imagery, J. Opt. Soc. America. A. 15(4), 978-992, (1998).
11. Li, D., Merserau, R. M., Blur identification based on kurtosis minimization. In: Proc, IEEE International Conference on Image Processing (ICIP). Vol. 1. Genova, Italy, pp. 905-908, September (2005).
12. Barney-Smith, E. H., PSF estimation by gradient descent fit to the ESF, Proc. SPIE Electronic Imaging, Image Quality and System Performance III. Vol. 6059. San Jose, Calif., January (2006).
13. Dror, I., Kopeika, N. S, Experimental comparison of turbulence MTF and aerosol MTF through the open atmosphere, J. Opt. Soc. Am. A 12(5), 970-980, (1995).
14. Haik, O., Lior, Y., Nahmani, D., Yitzhaky, Y., Effects of image restoration on acquisition of moving objects from thermal video sequences degraded by the atmosphere, Opt. Eng. 45, 117006, (2006).
15. Reichenbach, S. E., Characterizing digital image acquisition, Opt. Eng. 30(2), 170-177, (1991).
16. Wong, H., Effect of Knife-Edge skew on modulation transfer function measurements of charge coupled device imagers employing a scanning Knife-Edge, Opt. Eng. 30(9), 1394-1398, (1991).
17. Yitzhaky, Y., Dror, I., Kopeika, N. S., Restoration of atmospherically-blurred images according to weather-predicted atmospheric modulation transfer function (MTF), Opt. Eng. 36(11), 3064-3072, (1997).
18. Canny, J. F., A Computational Approach to Edge Detection. IEEE Trans, Pattern Anal. Mach. Intell. 8(6), 679-698, (1986).
19. Jain, A. K., Fundamentals of image processing, Prentice Hall, Engle-wood Cliffs, N.J., (1989).
20. Koren, R., Yitzhaky, Y., Automatic selection of edge detector parameters based on spatial and statistical measures, Computer Vision and Image Understanding 102(2), 204-213, (2006).
21. Yitzhaky, Y., Peli, E., A method for objective edge detection evaluation and detector parameter selection, IEEE Trans. Pattern Analysis & Machine Intelligence 25(8), 1027-1033, (2003).
22. Darlington, R. B., Is kurtosis really peakedness?, American Statistician 24, 19-22, April, (1970).
23. Moors, J. J. A., The meaning of kurtosis, Darlington revisited, American Statistician 40, 283-284, (1986).
24. Palmer, A. R., Strobeck, C., Fluctuating asymmetry analyses revisited, Polak, M. (Ed.), Developmental instability: causes and consequences. Oxford University Press, New York, pp. 279-319, (2003).
25. Yitzhaky Yitzhak Home Page at Ben-Gurion University, Israel [web page], URL [Accessed 5 May 2007]: http://www.ee.bgu.ac.il/˜itzik/BestStep/
26. CONTROP precision technologies LTD. URL [Accessed 21 November 2006]: http://www.controp.co.il
27. Biggs, D. S. C., Andrews, M., Acceleration of iterative image restoration algorithms, Appl. Opt. 36(8), 1766-1775, (1997).
28. Loyev, V., Yitzhaky, Y., Initialization of iterative parametric algorithms for blind deconvolution of motion-blurred images, Appl. Opt. 45(11), 2444-2452, (2006).
29. Oppenheim, A. V., Schafer, R. W., Discrete-Time Signal Processing, Prentice-Hall, Engle-wood Cliffs, N.J., pp. 447-448, (1989).
Which are incorporated by reference for all purposes as if fully set forth herein.
Note that the inventors of the present application have hardcopies of all references mentioned as web URLs with the intention of preserving them even in case that the publications are removed from the internet.
Remotely sensed images captured by high-resolution imagers are likely to be degraded by the atmosphere [1]. The degradation sources which include turbulence and aerosols in the atmosphere cause image blur, in addition to spatio-temporal movements and intensity scintillations.
A system approach for restoration of atmospherically blurred images models the blurred image as a convolution between the original image and the atmospheric PSF. In such a case, simple deconvolution techniques (such as Wiener filtering) can be used to restore the image [1]. However, the main difficulty is the need for a reliable knowledge of the PSF. The reason for this difficulty is that in most practical remote sensing situations this data is not known a-priori. Usually the only available data is the recorded image itself. An image deconvolution process in this case is called blind deconvolution or blind restoration.
There has been extensive work on blind deconvolution over the past twenty-one years, including survey articles [2, 3]. Existing blind de-convolution methods can be categorized into two main classes: methods which separate blur (PSF) identification as a disjoint procedure from restoration [4-6], and methods which combine blur identification and restoration in one procedure [7-10]. Methods in the first class tend to be computationally simpler. For example, Chalmond [5] proposed to isolate specific features in the image, such as sharp edge elements, and then to estimate the PSF from them, assuming radial symmetry. However, a main drawback of this method is the assumption that the shapes of all the extracted edges can be modeled as ideal step functions in the original (un-blurred) image. No criterion is employed to evaluate the best (closest to ideal) step-edge from the set of sharp edge elements in the degraded image.
Methods in the second class usually use iterative procedures to estimate the blur extent [2, 3, 7-10]. They often formulate parametric models for both the image and the blur, and in each step the image and the blur parameters are estimated and used in the next iteration. A Gaussian function is often used to approximate the atmospheric PSF [8-10]. A shortcoming of these methods is the requirement for a good initial guess of the PSF. The resulting accuracy of the estimated PSF (and consequently the quality of the restored images) depends on the accuracy of this initial guess. Also, the algorithm speed of convergence depends on this guess.
Recently, a blind deconvolution method based on kurtosis minimization has been proposed [11]. Using different choices for the blur parameters, the noisy blurred image is restored with a Wiener filter. Then, a statistical criterion (minimum kurtosis) is used as a measurement for the quality of the restored images. However, one of the disadvantages of this method is the requirement for a-priori knowledge of the blur parameter range for a reasonable computational load.